Web Survey Bibliography
Title Careless Response and Attrition as Sources of Bias in Online Survey Assessments of Personality Traits and Performance
Source Computers in Human Behaviour; 76, pp. 417-430
Year 2017
Database ScienceDirect
Access date 20.07.2017
Abstract The online survey is critical to organizations to efficiently collect large amounts of data about variables like performance. However, this data collection method comes with challenges. Careless responding (CR) and attrition in online surveys introduce measurement error, and can lead to several psychometric issues. Despite the common co-occurrence of CR and attrition, previous studies have looked at CR and attrition separately, which may miss their additive impact in online survey measurement. Additionally, research has focused on CR and attrition without attending to their relationships with other variables. The aims of our studies are to build the nomological network around CR and attrition, and to determine the effects of the co-occurrence of CR and attrition. We investigated (a) the extent to which personality traits relate to both CR and attrition, and (b) the extent to which CR and attrition affect estimates of the relationships between personality traits and performance. We found a relatively high base rate of carelessness (23%) and mid-study attrition (11%). Conscientiousness, extraversion, neuroticism, and agreeableness were related to survey attrition and carelessness resulting in significant differences between respondents who carefully completed the survey and those lost via attrition or careless response screening. A simulation study provided estimates of the extent of bias resulting from: 1) various amounts of attrition and carelessness, and 2) correlations between attrition and carelessness with personality traits. Although the magnitude of bias is modest in most cases, there was substantial bias (.15) in correlation estimates in some situations when screening for CR and attrition. Based on findings from the current studies and extant literature, we suggest ways researchers and practitioners can address CR and attrition to improve the accuracy of the measured relationships between variables like personality and performance, and enhance the defensibility of conclusions.
Access/Direct link Journal Homepage (abstract) / (full text)
Year of publication2017
Bibliographic typeJournal article
Web survey bibliography - Computers in Human Behaviour (42)
- Careless Response and Attrition as Sources of Bias in Online Survey Assessments of Personality Traits...; 2017; Meade, A. W.; Ward, M. K.; Alfred, C. M.; Pappalardo, G.; Stoughton, J. W.
- Web survey experiments on matrix questions; 2017; Liu, M.
- Facebook, Twitter, & Qr codes: An exploratory trial examining the feasibility of social media mechanisms...; 2016; Gu, L. L.; Skierkowski, D.; Florin, P.; Friend, K.; Ye, Y.
- On-line life history calendar and sensitive topics: A pilot study; 2016; Morselli, D.; Berchtold, A.; Granell, J.-C. S.; Berchtold, And.
- The impact of visual design and response formats on data quality in a web survey of MOOC students; 2016; Maloshonok, N.; Terentev, E.
- Development of a scale to measure skepticism toward electronic word-of-mouth; 2016; Zhang, Xia.; Ko, M.; Carpenter, D.
- Improving social media measurement in surveys: Avoiding acquiescence bias in Facebook research; 2016; Kuru, O.; Pasek, J.
- Psychological research in the internet age: The quality of web-based data; 2016; Ramsey, S. R.; Thompson, K. L.; McKenzie, M.; Rosenbaum, A.
- Internet Abusive Use Questionnaire: Psychometric properties; 2016; Calvo-Frances, F.
- Equivalence of paper-and-pencil and computerized self-report surveys in older adults; 2016; Weigold, A.; Weigold, I. K.; Drakeford, M. K.; Dykema, S. A.; Smith, C. A.
- Investigating respondent multitasking in web surveys using paradata; 2016; Sendelbah, A.; Vehovar, V.; Slavec, A.; Petrovcic, A.
- The effect of email invitation elements on response rate in a web survey within an online community; 2016; Petrovcic, A.; Petric, G.; Lozar Manfreda, K.
- A reliability analysis of Mechanical Turk data; 2016; Rouse, S. V.
- Validation of the new scale for measuring behaviors of Facebook users: Psycho-Social Aspects of Facebook...; 2015; Bodroza, B.; Jovanovic, T.
- An experiment testing six formats of 101-point rating scales; 2015; Liu, M.; Conrad, F. G.
- Doing online research involving university students with disabilities: Methodological issues; 2015; De Cesarei, A.; Baldaro, B.
- Data collection mode effect on feeling thermometer questions: A comparison of face-to-face and Web surveys...; 2015; Liu, M., Wang, Yi.
- Using virtual presence and survey instructions to minimize careless responding on Internet-based surveys...; 2015; Ward, M. K.; Pond III, S. B.
- The comparison of mobile devices to computers for web-based assessments; 2015; Huff, K. C.
- Differences in intrapersonal and interactional empowerment between lurkers and posters in health-related...; 2014; Petrovcic, A., Petric, G.
- Social desirability is the same in offline, online, and paper surveys: A meta-analysis; 2014; Dodou, D., de Winter J. C. F.
- Completion rates and non-response error in online surveys: Comparing sweepstakes and pre-paid cash incentives...; 2014; LaRose, R., Tsai, H. S.
- Beauty is more than screen deep: Improving the web survey respondent experience through socially-present...; 2014; Casey, T. W., Poropat, A.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- Comparison of web-based versus paper-and-pencil administration of a humor survey; 2013; Wang, C.-C., Cheng, C.-L., Cheng, Y.-Y., Liu, K.-S.
- Does survey format influence self-disclosure on sensitive question items?; 2012; Kays, K., Gathercoal, K., Buhrow, W.
- Measurement invariance in training evaluation: Old question, new context; 2011; P., Gissel, A., Stoughton, J. W., Whelan, T. J.Clark, A. P.
- Color red in web-based knowledge testing; 2010; Gnambs, T., Batinic, B., Appel, Ma.
- Internet administration of self-report measures commonly used in research on social anxiety disorder...; 2010; Hedman, E., Ljótsson, B., Rück, C., Furmark, T., Carlbring, P., Lindefors, N., Andersson, G.
- Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): Participation...; 2010; Denniston, M. M., Brener, N. D., Kann, L., Eaton, D. K., McManus, T., Kyle, T. M., Roberts, A. M., Flint...
- Factors affecting response rates of the web survey: A systematic review; 2010; Fan, W., Yan, Z.
- Psychometric properties of internet administered versions of Penn State Worry Questionnaire (PSWQ) and...; 2009; Zlomke, K. R.
- Web-based measurement: Effect of completing single or multiple items per webpage; 2009; Thorndike, F. P., Carlbring, P., Smyth, F. L., Magee, J. C., Gonder-Frederick, L., Ost, L. G., Ritterband...
- From paper to pixels: A comparison of paper and computer formats in psychological assessment ; 2008; Naus, M. J., Philipp, L. M., Samsi, M.
- Personalized salutation, power of sender and response rates to Web-based surveys; 2007; Joinson, A. N., Reips, U. -D.
- Computer-assisted questionnaires may facilitate collection of quality-of-life (QOL) data: At a cost; 2006; Smith, Ad. B., Velikova, G., Wright, E. P., Lynch, P., Selby, P. J.
- Exploring users' attitudes and intentions toward the web as a survey tool; 2005; Huang, H.-M., Liaw, S. S.
- Do print and Web surveys provide the same results?; 2004; Huang, H.-M.
- Humanizing self-administered surveys: experiments on social presence in web and IVR surveys; 2003; Tourangeau, R., Couper, M. P., Steiger, D. M., de Rouvray, C.
- Using pencil and paper, Internet and touch-tone phones for self-administered surveys: does methodology...; 2003; Knapp, H., Kirk, S. A.
- Using touch screen audio-CASI to obtain data on sensitive topics; 2001; Cooley, P. C., Rogers, S. M., Al-Tayyib, A. A., Ganapathi, L. F., Willis, G. B., Turner, C. F.